1. Field of the Invention
The present invention relates to encoding and decoding processes of moving images.
2. Description of the Related Art
International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 13818-2 (hereinafter, “Moving Picture Experts Group (MPEG) 2”) and International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) Recommendation H.264 (hereinafter, “H.264”), both of which are widely known as an international standard for moving image encoding processes, define an image frame or an image field, each of which is a unit of compression, as a “picture”. Each “picture” is used as an access unit in encoding and decoding processes. In a normal encoding process, the code amount fluctuates for each of “pictures” depending on complexity of the image and the encoding mode being used (e.g., an intra-frame encoding mode, a forward prediction encoding mode, a bi-directional encoding mode).
To realize transmission and playback processes without problems while using a transmission channel having a fixed bit rate or a transmission channel for which the maximum transmission rate is determined, each of these international standards defines a virtual decoder model and prescribes that it is mandatory for an encoder to control the code amount fluctuation in units of “pictures” in such a manner that no overflow or underflow occurs in a reception buffer model of a virtual decoder. The virtual decoder model is called a Video Buffering Verifier (VBV) according to MPEG-2 and is called a Hypothetical Reference Decoder (HRD) according to H.264. A virtual reception buffer is called a VBV buffer in the VBV model and is called a Coded Picture Buffer (CPB) in the HRD model. In these virtual reception buffer models, operations that use “pictures” as access units are defined (hereinafter, the terms “picture” and “pictures” will be used without the quotation marks).
According to MPEG-2 and H.264, a total delay amount between a time at which a moving image signal is input and a time at which the moving image signal is compressed and transmitted, and then, decompressed and displayed on the reception side is normally at least hundreds of milliseconds to a number of seconds. It means a delay corresponding to a number of image frames (up to tens of image frames) occurs. For this reason, it is essential to realize low-delay processing in various usages that require immediacy, such as real-time image communications or video games.
In the VBV model according to MPEG-2 that is defined in ISO/IEC 13818-2 and the HRD model according to H.264 that is defined in ITU-T Recommendation H.264, a low-delay mode is provided in addition to a normal-delay mode (cf. JP-A H08-163559 (KOKAI)). In these low-delay modes in the reception buffer models, if all the pieces of encoded data related to a picture are stored in the reception buffer at a picture decoding time, the decoding process is started. On the contrary, if all the pieces of encoded data related to the picture have not yet been stored in the reception buffer at the picture decoding time, the decoding process is skipped, so that the picture is decoded and displayed at another picture decoding time immediately after all the pieces of encoded data related to the picture have been stored in the reception buffer.
The encoder calculates the number of skipped frames using the virtual reception buffer model and discards as many input pictures as the number of skipped frames that has been calculated. The low-delay models according to MPEG-2 and H.264 manage the virtual buffers in units of pictures. Thus, a compression/decompression delay corresponding to at least one picture occurs.
Also, for the HRD model according to H.264, a buffer model that simultaneously satisfies transmission models having a plurality of transmission bandwidths with respect to the same compressed data has been defined (cf. JP-A 2003-179665(KOKAI) and JP-A 2007-329953(KOKAI)). In the transmission models having the plurality of bandwidths, it is mandatory that an encoding process is performed so that the data is transmitted without any underflow or overflow in all of the plurality of transmission bandwidths. As the transmission bandwidth becomes larger, it is possible to reduce the transmission/reception buffer delay time period and to shorten the compression/decompression delay.
However, because it is necessary to guarantee the transmission in the transmission model having the smallest transmission bandwidth among the plurality of bandwidths, it is not possible to improve the image quality by effectively utilizing the bandwidths in the transmission channels having larger transmission bandwidths. In addition, like in the conventional reception buffer model according to MPEG-2 or the like, because buffer control is exercised in units of pictures, there is a limit to how much compression/decompression delay can be lowered.
As explained above, in the conventional moving image encoding methods according to MPEG-2, H.264, and the like, the transmission buffer delay caused by the virtual buffer model operating in units of pictures also occurs in addition to the delays in the encoding process and the decoding process. Thus, a large display delay occurs in real-time image transmission that involves compressions and decompressions. Furthermore, in the conventional low-delay modes, problems remain where frame skipping occurs and where it is necessary to use a transmission channel having a larger bandwidth than required by an encoded data amount.